Overview

• Introduction

  • Data
  • Exploratory Data Analysis

• Methodology & Analysis

  • Logistic Models
  • Perceptron Models
  • Decision Tree Models
  • ROC Curves
  • Bagging

• Results & Conclusion

• General Discussion

Introduction

• Telecommunication customer service data

  
  

• 1,000 customer observations

  
  

• 14 variables

Variables

Distribution of Churn

• A majority of churn observations are no, meaning a majority of customers are not churning
• Still a 3:1 ratio of no churn:churn
• Plenty of data to work with

Continuous Variable Distribution by Churn

• Churned customers have higher monthly charges, and shorter terms
• Total charges shows less of a difference between churners and non churners

Logistic Regression

• Three logistic models were evaluated:

  • Full model

  • Reduced model

  • Stepwise-selected model

• Models were compared using ROC curves and AUC on a held-out test set.

Full Model

• Significant predictors were term, internet service no internet, agreement period one year and two year, and total charges.
• Some factor levels produced NA estimates due to multicollinearity but did not harm the model.

Reduced Model

• Significant predictors were term, internet service no internet, agreement period one and two year, and total charges

Stepwise Logistic Results

• Significant predictors were term, technical support no internet, agreement period one and two year, and total charges.

Full Model

• Includes all the variables. Numeric variables were scaled, categorical variables were converted to dummy variables
• Weights indicate that streaming videos (no internet), monthly charges, internet service (no service), Multiple lines (No), and Phone Service (Yes) have strong influence

Reduced Model

• Includes most important predictors from the full model
• Strong influence from term and total charge
• Moderate influence from agreement period

Non-Penalized Decision Tree

• Non-penalize model, no deductions for false positives or false negatives

Penalized Decision Tree (False Negatives)

• A false negative would mean that the model incorrectly identifies a customer as a churn risk when they would actually stay loyal

Penalized Decision Tree (False Positives)

• A false positive would mean that the model incorrectly identifies a customer as loyal when they actually leave

Logistic ROC

• Stepwise narrowly beats Full which narrowly beats Reduced
• Stepwise is the ideal final logistic regression model

              Model       AUC
1     Full Logistic 0.8042817
2  Reduced Logistic 0.7915127
3 Stepwise Logistic 0.8081395

Perceptron ROC

• Perfect AUC for the full model, most likely an error, attempts to correct it were unsuccessful
• Leads to the reduced model being the optimal perceptron model

               Model       AUC
1    Full Perceptron 1.0000000
2 Reduced Perceptron 0.7410074

Decision ROC

• There are 6 curves cause each model has a gini and info curve
• The decision tree that penalized false negatives has the strongest results

Bagging

• Now each of the final models need to be compared
• Logistic and Decision Tree models are similar and strong
• Perceptron model is the weakest of the three models

Results & Conclusions

• Final optimal model is the stepwise logistic regression

• Due to the smaller AUC and more stability

• Good amount of variable overlap with other models

\[ \text{Churn} = -0.5002 - 0.0758(\text{Term}) - 1.309(\text{TS_NoInternet}) - 0.451(\text{TS_Yes}) \\ - 1.612(\text{OneYear}) - 2.146(\text{TwoYear}) \\ + 0.01183(\text{MonthlyCharges}) + 0.000642(\text{TotalCharges}) \]

General Discussion

• Potential faults

  • Argument for Decision Tree potentially being better

• Stepwise model might be harder to interpret to non statisticians

• If someone does not understand how to implement a model its useless to them